The blood of humans contains red blood cells, platelets and white blood cells. In medical applications there is often an interest in classifying the white blood cells into approximately twenty different sub classes and in estimating the distribution of these subclasses for individual patients. Traditionally this classification may be performed in a laborious way by a human expert looking through the eyepiece of a brightfield microscope and thereby studying blood which has been wedged on a slide. The classification may also, with less separable sub classes though, be performed in a so called cell counter where diluted blood passes, cell by cell, through a narrow strait in a so called flow cell. While the cells pass through the flow cell parameters like electrical impedance and light scattering are measured, which subsequently are the basis for the classification of the actual white blood cell performed by the cell counter.
The advances within image analysis during recent years have made it possible to produce automatical brightfield microscopes that are capable of scanning a blood smear for white blood cells, autofocus on them, store well focused cell images, perform automatical segmentation of the resulting cell images, calculate so called features for each segmented cell image and finally classify each found white blood cell based on the information in the calculated features.
Segmentation is the process of splitting the contents of an image into different regions. In the case of white blood cells, the desired result is that white blood cells are separated from the other objects in the image like background, red blood cells, platelets, garbage and, of course, from other white blood cells. It may also be desirable that the segmentation separates the nucleus of the white blood cell from its cytoplasm.
The main reason for segmenting the individual cells and their parts is that there, today, do not exist any classifaction methods that work well enough and quickly enough without the separation of image information that the segmentation corresponds to.
Acceptable segmentation results may sometimes be obtained by using so called binary thresholding, meaning that each image element, also called pixel, in the cell image is dealt with depending on if its intensity is greater than or below a threshold intensity. The threshold intensity may be local or global, i.e. constant over the whole image.
An apparent drawback of binary thresholding is that the object contour that is obtained is sensitive to the level of the chosen threshold intensity, for example when the illumination conditions vary within av cell image. In addition, thresholding is performed regardless of shape or so called non-local information, i.e. how the object appears at some distance from the sought contour. Binary thresholding may therefore give the result that adjacent objects are not separated but segmented as one object or that an object with an original smooth, rounded contour gets a zigzag contour after segmentation. Binary thresholding may also give the result that a single original object is segmented as multiple objects. Binary thresholding and more sophisticated thresholding methods are therefore most often only used as initial operations in more powerful segmentation methods.
So called active contour models is a class of methods that have been used for quite a while for segmentation of images. Using the active contour models the contour of the object is determined directly instead of indirectly via which image elements that belong to the object, like it is done using thresholding. All active contour models use some kind of contour that iteratively, during the process of segmentation, is altered under the influence of so called internal forces that originate from the contour and its shape and under the influence of so called external forces that originate from the image and mainly from its contents of edge information.
One way of enhancing edge information in images is to calculate so called gradient images. Briefly that means that elements in the image, where the intensity is increased or decreased with a large amount on a distance corresponding to a few image elements, get correspondingly large values at the corresponding image element positions in the gradient image. Edges are a good example of objects that give rise to large gradient values. The gradient images may then be post-processed in order to get rid of isolated large values, that probably were caused by garbage or small details, and in order to enhance long joint streaks of large values probably caused by edges of the sought object and other objects. An image where the values of the image elements consist of some kind of gradient values will be called an edge image below.
The active models may be divided into two groups. The first group is the group of geometrical models like “Level Sets” and “Fast Marching” methods. The contour is represented by the set of image elements of the contour. The advantage of that representation is that the contour may have arbitrary shapes. The iterations comprise operations on the image elements of the contour and their corresponding closest neighbours. Since the contour may have arbitrary shapes, it is relatively complicated and computationally heavy to administrate all the image elements of the contour, their neighbours and their matual relative positions.
The second group is the group of parametrical models like “snakes”, see the paper “Snakes: Active contour models,” International Journal of Computer Vision, 1(4): 321-331, 1987, by Kass, Witkin och Terzopoulos. A snake is a parametric contour model. The parameters in the model may for example consist of the positions of the corners of a polygon, but most often the model is more advanced, for example by consisting of joint curved contour elements. The internal forces of the snake strive for a smooth contour while the external forces strive for letting the contour pass close to many of the image elements that contain edge information. During the process of segmentation, an algorithm based on snakes will itererate position and shape of the snake until a good compromise—hopefully the most fitting contour—between internal and external forces is found. By increasing the influence of the internal forces, the shape of the final contour estimate may become more or less limited to circular shaped objects. By correspondingly increasing the influence of the external forces, the shape of the final contour estimate may become more irregular. Since the snake is represented by a limited number of parameters, its possible shapes are limited. For white blood cells, which have fairly regular contours, this limitation is seldom a drawback. On the contrary, it is an advantage that the snake will have a regular shape even in those cases where the contour of the object is seen only vaguely in the cell image and where it therefore will be quite a distance between image elements having edge information. In addition, the limited number of parameters will lead to that the computational power needed during iterations will be less compared to that of a geometrical model.
If the snake is started close to the sought contour, it will probably converge safely and with a small number of iterations. Such a start may be easy for a human operator but harder to accomplish using automatic image analysis. Automatic segmentation will therefore require that the snake may be started relatively far away from the sought object contour but still converge safely and quickly. That requirement will in turn lead to the requirement that the external forces of the snake must be able to lead the snake correctly even from image elements that are situated far away from the sought contour. In order to be useful as input to a snake, the external forces should simply, for each image element in the original image, be pointing towards image elements with a greater amount of edge information, i.e. with greater edge resemblance. Therefore it is suitable to use the expression “vector force field” for the external forces as a function of the location in the cell image.
One previously known way of obtaining a vector force field is to start with an edge image with values f(x,y), that depend on the x and y coordinate of the corresponding image element and calculate differences in f(x,y) with respect to x and y. The calculations of differences may for example be defined byf—x(x,y)=f(x+1,y)−f(x−1,y)  (eq. 1) andf—y(x,y)=f(x,y+1)−f(x,y−1)  (eq. 2), respectively.
The obtained vector force field [f_x(x,y),f_y(x,y)] is an example of an external force field. The greatest drawback of such a vector force field is that the amount of force rapidly descends as the coordinates (x,y) move away from those points that have an edge image f(x,y) that differs from zero. Thus, such a vector force field does not work very well when the snake is started from a distance relatively far away from the sought object contour.
One previously known way of improving the ability of the vector force field to act at a distance is to smear the edge image, before or after the calculations of differences with for example a two-dimensional Gaussian filter, but that may result in other problems like details that become less apparent.
The Gradient Vector Flow Method is a previously know method that accomplishes a static external vector force field, a so called GVF field (Gradient Vector Flow Field), calculated on the basis of a, like shown above, difference calculated edge image containing information on the sought object contour. (See for example the Master's thesis “Segmentation of Histopathological Tissue Sections Using Gradient Vector Flow Snakes”, Centre for Mathematical Sciences, Lund University and Lund Institute of Technology, Mar. 18, 2002, by Adam Karlsson or the original reference “Gradient vector flow: A new external force for snakes,” IEEE Proceedings on Computer Vision and Pattern Recognition, Puerto Rico, pages 66-71, 1997 by Xu och Prince).
A GVF-field fulfills the requirement of being an external vector force field that leads the snake in the correct direction also at large geometrical distances from the sought contour. A snake that uses a GVF-field is called a GVF-snake below. The GVF-field is then calculated once per segmentation—at the beginning of it. Such segmentation methods are previously known.
In the Master's thesis above by Adam Karlsson there is further described a fast way of iterating the parameters of the snake, i.e. how the position and the shape of the snake is iterated based on the GVF-field. In the thesis it is pointed out that the timing performance of a GVF-snake, which uses this fast way of iterating the parameters, will be limited by the time it takes to calculate the GVF-field.
The traditional way of calculating the GVF field according to Xu och Prince contains iterative solving ofμΔu−(u−f—x)(f—x2+f—y2)=0  (eq. 3) andμΔv−(v−f—x)(f—x2+f—y2)=0  (eq. 4),where Δ is the so called Laplace operator—a kind of two-dimensional second order difference operator—and where μ is a parameter that may have to be tuned depending on the application, i.e. what kind of objects it is and in which environment they are situated.
The calculation of the GVF-field means that a vector force field [f_x(x,y),f_y(x,y)] with limited ability to act at a distance results in another vector force field—the GVF-field, [u(x,y),v(x,y)]—with an improved ability to act at a distance without the loss of details. The input to the calculation can be regarded as two images, f_x and f_y respectively, and the output can be regarded as two other images, u and v respectively.
Solving equation systems 3 and 4 requires a considerable amount of iterations in order to get a converged result. If the area of interest in the original image has the size of m times n pixels, each of the equation systems 3 and 4, will contain m times n equations, which means just as many unknowns, which for images easily becomes ten thousand unknowns which shall be updated during each iteration.
It is not unusual that the traditional way of calculating the GVF-field, with a 1 GHz PC-processor, will need several tenths of a second per cell image. For an analysis of 200 white blood cells that corresponds to a total segmentation time of close on one minute, which limits the performance of existing automatic brightfield microscopes unless extra, expensive hardware is added.